Generating bridge geometric digital twins from point clouds

The automation of digital twinning for existing bridges from point clouds remains unsolved. Extensive manual effort is required to extract object point clusters from point clouds followed by fitting them with accurate 3D shapes. Previous research yielded methods that can automatically generate surface primitives combined with rule-based classification to create labelled cuboids and cylinders. While these methods work well in synthetic datasets or simplified cases, they encounter huge challenges when dealing with realworld point clouds. In addition, bridge geometries, defined with curved alignments and varying elevations, are much more complicated than idealized cases. None of the existing methods can handle these difficulties reliably. The proposed framework employs bridge engineering knowledge that mimics the intelligence of human modellers to detect and model reinforced concrete bridge objects in imperfect point clouds. It directly produces labelled 3D objects in Industry Foundation Classes format without generating low-level shape primitives. Experiments on ten bridge point clouds indicate the framework achieves an overall detection F1-score of 98.4%, an average modelling accuracy of 7.05 cm, and an average modelling time of merely 37.8 seconds. This is the first framework of its kind to achieve high and reliable performance of geometric digital twin generation of existing bridges

Digital twinning of existing bridges from labelled point clusters

The automation of digital twinning for existing bridges from point clouds has yet been solved. Whilst current methods can automatically detect bridge objects in points clouds in the form of labelled point clusters, the fitting of accurate 3D shapes to detected point clusters remains human dependent to a great extent. 95% of the total manual modelling time is spent on customizing shapes and fitting them to right locations. The challenges exhibited in the fitting step are due to the irregular geometries of existing bridges. Existing methods can fit geometric primitives such as cuboids and cylinders to point clusters, assuming bridges are made up of generic shapes. However, the produced geometric digital twins are too ideal to depict the real geometry of bridges. In addition, none of existing methods have evaluated the resulting models in terms of spatial accuracy with quantitative measurements. We tackle these challenges by delivering a slicing-based object fitting method that can generate the geometric digital twin of an existing reinforced concrete bridge from labelled point clusters. The accuracy of the generated models is gauged using distance-based metrics. Experiments on ten bridge point clouds indicate that the method achieves an average modelling distance smaller than that of the manual one (7.05 cm vs. 7.69 cm) (value included all challenging cases), and an average twinning time of 37.8 seconds. Compared to the laborious manual practice, this is much faster to twin bridge concrete elements

Challenges of bridge maintenance inspection

Bridges are amongst the largest, most expensive and complex structures, which makes them crucial and valuable transportation asset for modern infrastructure. Bridge inspection is a crucial component of monitoring and maintaining these complex structures. It provides a safety assessment and condition documentation on a regular basis, noting maintenance actions needed to counteract defects like cracks, corrosion and spalling. This paper presents the challenges with existing bridge maintenance inspection as well as an overview on proposed methods to overcome these challenges by automating inspection using computer vision methods. As a conclusion, existing methods for automated bridge inspection are able to detect one class of damage type based on images. A multiclass approach that also considers the 3D geometry, as inspectors do, is missing

Detection of key components of existing bridge in point cloud datasets

The cost and effort for modelling existing bridges from point clouds currently outweighs the perceived benefits of the resulting model. Automating the point cloud-to-Bridge Information Models process can drastically reduce the manual effort and cost involved. Previous research has achieved the automatic generation of surfaces primitives combined with rule-based classification to create labelled construction models from point clouds. These methods work very well in synthetic dataset or idealized cases. However, real bridge point clouds are often incomplete, and contain unevenly distributed points. Also, bridge geometries are complex. They are defined with horizontal alignments, vertical elevations and cross-sections. These characteristics are the reasons behind the performance issues existing methods have in real datasets. We propose to tackle this challenge via a novel top-down method for major bridge component detection in this paper. Our method bypasses the surface generation process altogether. Firstly, this method uses a slicing algorithm to separate deck assembly from pier assemblies. It then detects pier caps using their surface normal, and uses oriented bounding boxes and density histograms to segment the girders. Finally, the method terminates by merging over-segments into individual labelled point clusters. Experimental results indicate an average detection precision of 99.2%, recall of 98.3%, and F1-score of 98.7%. This is the first method to achieve reliable detection performance in real bridge datasets. This sets a solid foundation for researchers attempting to derive rich IFC (Industry Foundation Classes) models from individual point clusters

Digital twinning of existing reinforced concrete bridges from labelled point clusters

The automation of digital twinning for existing reinforced concrete bridges from point clouds remains an unresolved problem. Whilst current methods can automatically detect bridge objects in point clouds in the form of labelled point clusters, the fitting of accurate 3D shapes to point clusters remains largely human dependent largely. 95% of the total manual modelling time is spent on customizing shapes and fitting them correctly. The challenges exhibited in the fitting step are due to the irregular geometries of existing bridges. Existing methods can fit geometric primitives such as cuboids and cylinders to point clusters, assuming bridges are comprised of generic shapes. However, the produced geometric digital twins are too ideal to depict the real geometry of bridges. In addition, none of the existing methods have explicitly demonstrated how to evaluate the resulting Industry Foundation Classes bridge data models in terms of spatial accuracy using quantitative measurements. In this article, we tackle these challenges by delivering a slicing-based object fitting method that can generate the geometric digital twin of an existing reinforced concrete bridge from four types of labelled point cluster. The quality of the generated models is gauged using cloud-to cloud distance-based metrics. Experiments on ten bridge point cloud datasets indicate that the method achieves an average modelling distance of 7.05 cm (while the manual method achieves 7.69 cm), and an average modelling time of 37.8 seconds. This is a huge leap over the current practice of digital twinning performed manually

Recursive segmentation for as-is bridge information modelling

Prior studies reported that the time needed to manually convert a point cloud to an as-is geometric model using cutting edge modelling software is ten times greater than the time needed to obtain the point cloud. The laborious nature of manually modelling infrastructure such as bridges is the reason behind the significant cost of modelling which impedes the proliferation of the usage of Bridge Information Models (BrIM) in Bridge Management Systems. Existing commercial solutions can automatically recognize geometric shapes embedded in segmented point cloud data (PCD) and generate the corresponding IFC objects. Researchers have taken further studies and have additionally automated surface reconstruction through generating parametric surface-based primitives in order to automate the segmentation process. However, surface-based segmentation for bridge modelling is an unsolved problem, which is neither straightforward nor consistent, thus hinders the automation of BrIM.This paper presents a top-down PCD detection solution that follows a knowledge-based heuristic approach for BrIM generation that can semi-automatically segment a bridge point cloud recursively. We leverage bridge domain knowledge as strong priors through a histogram-based algorithm to conduct the tasks of segmentation and classification. We implemented this solution and tested on one highway bridge. The experimental results indicated that the detection precision of this solution is 92%

3D reconstruction and measurement of surface defects in prefabricated elements using point clouds

Due to higher efficiency and lower cost, prefabricated construction is gradually gaining acceptance within the market. Laser scanning has already been adopted in civil engineering to reconstruct 3D model of structure and monitoring the deformation and so on. This paper seeks to explore a more automated and accurate quality control process focusing on the surface defects in prefabricated elements. Laser scanning is adopted for data collection and the 3D reconstruction of the prefabricated components. Besides, a new point cloud pre-processing, involving the KNN algorithm, reduction of data dimension and data gridding, is developed to improve the efficiency and accuracy of subsequent algorithms. The Delaunay triangle is used to extract the contour of the point cloud, then the contour is fitted to further determine the geometric data. Meanwhile, a comprehensive quality control system of prefabricated components based on relevant specifications is proposed, and the quality of prefabricated components is monitored intuitively by the values of indicators. In order to integrate into the BIM platform and better store the obtained quality information, the production quality information is designed to be extended to the IFC standard. The proposed approach will be applied to analyze the causes of quality problems in the production process and strengthen the quality control. This study designs a more efficient and accurate quality evaluation process, including data collection, data processing, indicator calculation and quality evaluation. Moreover, the results forward can feedback to the cause of the quality issues, and further improve the production quality of prefabricated elements

Multi-classifier for reinforced concrete bridge defects

Classifying concrete defects during a bridge inspection remains a subjective and laborious task. The risk of getting a false result is approximately 50% if different inspectors assess the same concrete defect. This is significant in the light of an over-aging bridge stock, decreasing infrastructure maintenance budgets and catastrophic bridge collapses as happened in 2018 in Genoa, Italy. To support an automated inspection and an objective bridge defect classification, we propose a three-staged concrete defect classifier that can multi classify potentially unhealthy bridge areas into their specific defect type in conformity with existing bridge inspection guidelines. Three separate deep neural pre-trained networks are fine-tuned based on a multi source dataset consisting of self-collected image samples plus several Departments of Transportation inspection databases. We show that this approach can reliably classify multiple defect types with an average mean score of 85%. Our presented multi-classifier is a contribution towards developing a mostly or fully inspection schema for a more cost-effective and more objective bridge inspection

Subsampling bias and the best-discrepancy systematic cross validation

Statistical machine learning models should be evaluated and validated before putting to work. Conventional k-fold Monte Carlo Cross-Validation (MCCV) procedure uses a pseudo-random sequence to partition instances into k subsets, which usually causes subsampling bias, inflates generalization errors and jeopardizes the reliability and effectiveness of cross-validation. Based on ordered systematic sampling theory in statistics and low-discrepancy sequence theory in number theory, we propose a new k-fold cross-validation procedure by replacing a pseudo-random sequence with a best-discrepancy sequence, which ensures low subsampling bias and leads to more precise Expected-Prediction-Error (EPE) estimates. Experiments with 156 benchmark datasets and three classifiers (logistic regression, decision tree and na¨ıve bayes) show that in general, our cross-validation procedure can extrude subsampling bias in the MCCV by lowering the EPE around 7.18% and the variances around 26.73%. In comparison, the stratified MCCV can reduce the EPE and variances of the MCCV around 1.58% and 11.85% respectively. The Leave-One-Out (LOO) can lower the EPE around 2.50% but its variances are much higher than the any other CV procedure. The computational time of our cross-validation procedure is just 8.64% of the MCCV, 8.67% of the stratified MCCV and 16.72% of the LOO. Experiments also show that our approach is more beneficial for datasets characterized by relatively small size and large aspect ratio. This makes our approach particularly pertinent when solving bioscience classification problems. Our proposed systematic subsampling technique could be generalized to other machine learning algorithms that involve random subsampling mechanism

An automated target-oriented scanning system for infrastructure applications

Point cloud pre-processing is essential for emerging applications such as digital twinning but currently requires a lot of manual effort before the resulting data can be used. Practitioners usually use default scan range settings to take full scans, which generate huge point cloud datasets containing millions of points. However, only a fraction of the dataset is used for subsequent twinning processes and the remaining data is “noise”. Researchers need to perform substantial cropping work to enable the point cloud can be used for detecting objects of interest. However, the problem of object detection in the post-processing stage also remains unresolved. This paper describes a new system TOSS to conduct a target-oriented scanning process. It streamlines the scan-to-gDT (geometric digital twin) process by automatically identifying the region of interest and its corresponding scanning path. TOSS consists of a cost-effective 3- DoF rotational laser scanner, a vision-based object detection algorithm, and a geometric-camera-model-based scanning control algorithm. Preliminary results on a real-world bridge indicate that TOSS can produce accurate scans of regions of interest (average: 95.5% Precision and 89.4% Recall). It is fully scalable and can be adapted to various infrastructure types, including buildings, bridges, industrial plants, tunnels, and roads. The algorithms also have great potential to be embedded in a traditional scanner’s software